feat: Add async big segment store manager and async Redis adapter#462
feat: Add async big segment store manager and async Redis adapter#462jsonbailey wants to merge 8 commits into
Conversation
Adds AsyncBigSegmentStoreManager, the async Redis-backed AsyncBigSegmentStore adapter, and Redis.async_big_segment_store(). Seeds async_config.py with the self-contained AsyncBigSegmentsConfig (the full AsyncConfig lands later). Bumps the redis dependency to >=4.2.0 (required by redis.asyncio).
Drop the max_connections=10 default (and the now-unneeded redis_opts copy) so the async store passes redis_opts straight to from_url, matching the sync store. The sync side removed this cap in #387 (it throttled connections under load); redis.asyncio otherwise defaults to an effectively-unlimited pool.
…egments_common Both big_segments.py and async_big_segments.py defined the class identically (bar one docstring word); move it to a shared module imported by both, matching the evaluator_common/client_common pattern.
Fold _hash_for_user_key, EMPTY_MEMBERSHIP, and is_stale (as a free function) into big_segments_common, imported by both big_segments.py and async_big_segments.py.
Drop the redundant EMPTY_MEMBERSHIP class attribute and reference the big_segments_common module constant directly instead of via self.
| [project.optional-dependencies] | ||
| async = ["aiohttp>=3.9,<4"] | ||
| redis = ["redis>=2.10.5"] | ||
| redis = ["redis>=4.2.0,<6.0.0"] |
There was a problem hiding this comment.
Is this not considered a breaking change?
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7bd5ebd. Configure here.
| status = self.__last_status | ||
| if status is None: | ||
| return BigSegmentStoreStatus(False, False) | ||
| return status if status else self.poll_store_and_update_status() # type: ignore[return-value] |
There was a problem hiding this comment.
get_status returns unawaited coroutine
Low Severity
In get_status, the fallback calls async poll_store_and_update_status without await, so that branch would return a coroutine object instead of a BigSegmentStoreStatus. The branch is unreachable with current BigSegmentStoreStatus objects, but it mirrors the sync manager incorrectly and is masked by a type ignore.
Reviewed by Cursor Bugbot for commit 7bd5ebd. Configure here.
2 participants
What
The async big-segments layer for the async Python SDK.
impl/async_big_segments.py—AsyncBigSegmentStoreManager(async twin of the sync manager: membership caching viaExpiringDict, status polling viaAsyncRepeatingTask) +BigSegmentStoreStatusProviderImpl.impl/integrations/redis/async_redis_big_segment_store.py— async Redis-backedAsyncBigSegmentStore(redis.asyncio).integrations/__init__.py— newRedis.async_big_segment_store(...)factory (local import of the adapter, soredisstays optional).async_config.py— seeded withAsyncBigSegmentsConfigonly (see below).testing/impl/test_async_big_segments.py(unit) +testing/integrations/test_async_redis.py(live-Redis integration, guarded byskip_database_tests).pyproject.toml— the published[redis]extra floor is unchanged (>=2.10.5); only the dev/testredisdependency is bumped to>=4.2.0so CI can exerciseredis.asyncio. Async Redis support requiresredis>=4.2.0, surfaced via a README note and the adapter's guarded import (a clear error ifredis.asynciois unavailable) rather than by raising the consumer-facing floor. Raising the published floor is deferred pending a team decision before the async client ships.Note on
async_config.pyAsyncBigSegmentStoreManagerneedsAsyncBigSegmentsConfig, which lives inasync_config.py. The fullAsyncConfigisn't extractable yet (it importsAsyncHook/AsyncPlugin/AsyncEventProcessor, which land in later slices), so this PR seedsasync_config.pywith just the self-containedAsyncBigSegmentsConfig(it depends only on the already-mergedAsyncBigSegmentStore). A later slice grows the file withAsyncConfig— mirroring how the sync SDK keepsBigSegmentsConfigalongsideConfiginconfig.py.Why
Part of the async Python SDK (epic SDK-60). Depends on the async support classes (#451) and async interfaces/store (#457), both merged.
Validation
uv run pytest ldclient/testing --ignore=ldclient/testing/integrations— 1033 passed, 2 skippeduv run pytest ldclient/testing/impl/test_async_big_segments.py— 14 passeduv run mypy ldclient— clean; isort / pycodestyle — cleantest_async_redis.py) require a live Redis (run in CI); properly guarded byskip_database_tests.SDK-2729 (epic SDK-60).
Note
Medium Risk
Changes affect flag evaluation inputs (Big Segment membership and HEALTHY/STALE/STORE_ERROR status) and add lifecycle/async polling behavior; marked experimental but correctness still matters for targeting.
Overview
Adds the experimental async Big Segments stack for the Python SDK:
AsyncBigSegmentsConfig,AsyncBigSegmentStoreManager(membership cache, background status polling viaAsyncRepeatingTask, inline poll when status is not ready yet, asyncstop), and an async RedisAsyncBigSegmentStoreexposed asRedis.async_big_segment_store().Sync code is refactored by moving
BigSegmentStoreStatusProviderImpl,EMPTY_MEMBERSHIP, user-key hashing, and stale checks intobig_segments_common.pyso the sync manager stays behaviorally the same with less duplication.async_config.pyis seeded with onlyAsyncBigSegmentsConfigahead of a laterAsyncConfigslice. Dev dependencyredisis bumped to>=4.2.0forredis.asyncio. Unit and live-Redis integration tests cover the new manager and adapter.Reviewed by Cursor Bugbot for commit 45a2125. Bugbot is set up for automated code reviews on this repo. Configure here.